COS 340: Reasoning About Computation∗ Sketching

The emergence of the web and the explosion of digital data of various forms has created the need for very efficient algorithms to deal with large data sets. A search engine is a good example of an application that needs to efficiently process very large amounts of data. Think about processing query logs to compute statistical information about the queries (e.g. the number of distinct queries), or processing the collection of documents crawled by the search engine to identify and eliminate near duplicate documents. Both these applications can be readily solved if the data set is relatively small, for example, if it fits in main memory. However the sheer size of the data set renders most common approaches infeasible. In the case of computing the number of distinct queries, the obvious technique that comes to mind is to make one pass over the query log and record the set of distinct queries seen so far (e.g. by storing them in a hash table). This can then be used to report the number of distinct elements. A search engine like Google probably receives close to a billion queries every day. The hash table approach is not a very practical solutions in this setting. How would you ever hope to scale this solution to compute the number of distinct queries in a year ? An emerging area in Computer Science is that of streaming algorithms, referring to very efficient algorithms that process very large amounts of data by making one or more passes over the data and recording very compact summaries, or sketches of the data. In constructing such compact summaries, we discard most of the data set, but retain some valuable information about it that still enables us to perform the computation we wanted. Ideally, we would like to be able to guarantee that such sketches allow us to compute the same things that could be computed with the entire data set. In most cases, this is too good to be true and we must settle for an approximation of what we wanted to compute. However, there is usually a tradeoff between the size of the sketch and the quality of the approximation it supports. In many cases of interest, by choosing our sketch sizes sufficiently large, we can ensure that the quantities of interest can be computed to any specified degree of accuracy.